Skip to content

[http1 codec] Allow requests with Transfer-Encoding and Content-Length headers set.#12349

Merged
mattklein123 merged 46 commits intoenvoyproxy:masterfrom
veshij:chunked
Aug 31, 2020
Merged

[http1 codec] Allow requests with Transfer-Encoding and Content-Length headers set.#12349
mattklein123 merged 46 commits intoenvoyproxy:masterfrom
veshij:chunked

Conversation

@veshij
Copy link
Contributor

@veshij veshij commented Jul 29, 2020

Commit Message: [http1 codec] Allow HTTP/1 requests/responses with Transfer-Encoding and Content-Length headers set.
Additional Description: opt-in for serving requests/responses with Content-Length and Transfer-Encoding: chunked. Per RFC remove Content-Length header before forwarding it to upstream.
Update http-parser version to include fix for nodejs/http-parser#517.
Risk Level: Medium
Testing: unit tests, update integration test
Docs Changes: updated docs/root/version_history/current.rst
Release Notes: http: added allow_chunked_length configuration option for HTTP/1 codec to allow processing requests with both Content-Length and Transfer-Encoding: chunked. If enabled - per RFC Content-Length is removed before sending to upstream.

Fixes #11398

veshij added 2 commits July 26, 2020 21:40
Copy link
Contributor

@alyssawilk alyssawilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, thanks for tackling this - I'm sure someone else down the line is going to have traffic which hits this unusual corner case.

Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for driving this forward. A few comments.

/wait

@antoniovicente
Copy link
Contributor

/assign @antoniovicente

Signed-off-by: Oleg Guba <oleg@dropbox.com>
@veshij
Copy link
Contributor Author

veshij commented Jul 29, 2020

/wait

Signed-off-by: Oleg Guba <oleg@dropbox.com>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/envoy/.
CC @envoyproxy/api-watchers: FYI only for changes made to api/envoy/.

🐱

Caused by: #12349 was synchronize by veshij.

see: more, trace.

Signed-off-by: Oleg Guba <oleg@dropbox.com>
@veshij
Copy link
Contributor Author

veshij commented Jul 30, 2020

/wait

@veshij
Copy link
Contributor Author

veshij commented Jul 30, 2020

/wait

Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
veshij added 2 commits July 30, 2020 13:11
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
mattklein123
mattklein123 previously approved these changes Aug 13, 2020
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this LGTM. Will give @PiotrSikora @htuch some more time to take a look again if they want. @antoniovicente is out till Monday so all things being equal I would prefer to wait for him to get back to take a final look. cc @asraa also for the H1 codec error handling changes.

Copy link
Contributor

@alyssawilk alyssawilk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good from my end - just two minor nits.

Copy link
Contributor

@antoniovicente antoniovicente left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looked at changes since my last set of comments. Thanks for adding tests for the response cases.

} else if (parser_.status_code == 204 || parser_.status_code == 304 ||
(parser_.status_code >= 200 && parser_.content_length == 0)) {
(parser_.status_code >= 200 && parser_.content_length == 0 &&
!(parser_.flags & F_CHUNKED))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity: what coverage do we have for 1xx responses that include content-length or transfer-encoding: chunked headers?

veshij added 3 commits August 24, 2020 12:17
…e code to use it

Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
@veshij
Copy link
Contributor Author

veshij commented Aug 26, 2020

/wait

veshij added 2 commits August 25, 2020 20:05
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
veshij added 3 commits August 26, 2020 11:15
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Oleg Guba <oleg@dropbox.com>
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks LGTM. @antoniovicente @alyssawilk any further comments?

@mattklein123 mattklein123 merged commit 954c93c into envoyproxy:master Aug 31, 2020
clarakosi pushed a commit to clarakosi/envoy that referenced this pull request Sep 3, 2020
…h headers set. (envoyproxy#12349)

opt-in for serving requests/responses with Content-Length and Transfer-Encoding: chunked. Per RFC remove Content-Length header before forwarding it to upstream.

Signed-off-by: Oleg Guba <oleg@dropbox.com>
Signed-off-by: Clara Andrew-Wani <candrewwani@gmail.com>
@ceastman-ibm
Copy link

When would this PR get into envoyproxy/envoy-wasm so that istio can pick it up? We are actually hitting this edge case. @alyssawilk

@alyssawilk
Copy link
Contributor

Up to folks on that side.
Did @jplevyak 's last merge sort this out for you? Looks like it landed the same day you pinged here.

@ceastman-ibm
Copy link

@alyssawilk we won't be able to test this until this version of envoyproxy gets into istio and then into our IBM managed istio offering.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request with Transfer-Encoding: chunked and Content-Length is valid per RFC, but rejected with HPE_UNEXPECTED_CONTENT_LENGTH

7 participants